File and Data Management with Qumulo API python bindings

This jupyter notebook walks through some of the basics of file and data management with Qumulo API python bindings.



In [68]:

    
import os
import re
import sys
import StringIO
from qumulo.lib.request import RequestError
from qumulo.rest_client import RestClient



In [69]:

    
# set your environment variables or fill in the variables below
API_HOSTNAME = os.environ['API_HOSTNAME'] if 'API_HOSTNAME' in os.environ else '{your-cluster-hostname}'
API_USER =     os.environ['API_USER']     if 'API_USER'     in os.environ else '{api-cluster-user}'
API_PASSWORD = os.environ['API_PASSWORD'] if 'API_PASSWORD' in os.environ else '{api-cluster-password}'



In [70]:

    
rc = RestClient(API_HOSTNAME, 8000)
rc.login(API_USER, API_PASSWORD)
print("logged in as: %(name)s" % rc.auth.who_am_i())









    



logged in as: admin

A few Qumulo API file and direcotory python bindings

fs.create_directory

arguments:

name: Name of directory to be created
dir_path*: Destination path for the parent of created directory
dir_id*: Destination inode id for the parent of the created directory

*Either dir_path or dir_id is required

fs.create_file

arguments:

name: Name of file to be created
dir_path: Destination path for the directory of created file
dir_id: Destination inode id for the directory of the created file

fs.write_file

arguments:

data_file: A python object of the local file's content
path: Destination file path on Qumulo
id_: Destination inode file id on Qumulo
if_match:

fs.get_attr

arguments:

path:
id_:
snapshot:

Create a working directory for this exercise



In [71]:

    
base_path = '/'
dir_name = 'test-qumulo-fs-data'

try:
    the_dir_meta = rc.fs.create_directory(dir_path=base_path, name=dir_name)
    print("Successfully created %s%s." % (base_path, dir_name))
except RequestError as e:
    print("** Exception: %s - Details: %s\n" % (e.error_class,e))
    if e.error_class == 'fs_entry_exists_error':
        the_dir_meta = rc.fs.get_attr(base_path + dir_name)

for k, v in the_dir_meta.iteritems():
    if re.search('(id|size|path|change_time)', k):
        print("%19s - %s" % (k, v))









    



Exception: fs_entry_exists_error - Details: Error 409: fs_entry_exists_error: { ino=2203379931 }

                 id - 2203379931
               size - 1024
               path - /test-qumulo-fs-data/
        change_time - 2017-05-02T19:24:26.493575055Z

Create a file in an existing path



In [73]:

    
file_name = 'first-file.txt'

# relies on the base path and direcotry name created in the code above.
try:
    the_file_meta = rc.fs.create_file(name=file_name, dir_path=base_path + dir_name)
except RequestError as e:
    print("** Exception: %s - Details: %s\n" % (e.error_class,e))
    if e.error_class == 'fs_entry_exists_error':
        the_file_meta = rc.fs.get_attr(base_path + dir_name + '/' + file_name)
print("We've got a file. Its id is: %s" % the_file_meta['id'])









    



** Exception: fs_entry_exists_error - Details: Error 409: fs_entry_exists_error: { ino=2206379876 }

We've got a file. Its id is: 2206379876



In [74]:

    
# writing a local file from /tmp/ to the qumulo cluster
fw = open("/tmp/local-file-from-temp.txt", "w")
fw.write("Let's write 100 sentences on this virtual chalkboard\n" * 100)
fw.close()

write_file_meta = rc.fs.write_file(data_file=open("/tmp/local-file-from-temp.txt"), 
                                   path=base_path + dir_name + '/' + file_name)

print("""name: %(path)s
bytes: %(size)s
mod time: %(modification_time)s""" % write_file_meta)









    



name: /test-qumulo-fs-data/first-file.txt
bytes: 5300
mod time: 2017-05-02T19:45:00.372716888Z



In [75]:

    
string_io_file_name = 'write-from-string-io.txt'

try:
    rc.fs.create_file(name=string_io_file_name, dir_path=base_path + dir_name)
except RequestError as e:
    print("Exception: %s - Details: %s\n" % (e.error_class,e))

fw = StringIO.StringIO()
fw.write("Let's write 200 sentences on this virtual chalkboard\n" * 200)
write_file_meta = rc.fs.write_file(data_file=fw, 
                                   path=base_path + dir_name + '/' + string_io_file_name)
fw.close()
print("""name: %(path)s
bytes: %(size)s
mod time: %(modification_time)s""" % write_file_meta)









    



Exception: fs_entry_exists_error - Details: Error 409: fs_entry_exists_error: { ino=2142379950 }

name: /test-qumulo-fs-data/write-from-string-io.txt
bytes: 10600
mod time: 2017-05-02T19:45:32.137036906Z



In [ ]: